Method of calibrating a patient monitoring system for use with a radiotherapy treatment apparatus

ABSTRACT

Some embodiments are directed to an image director of a patient monitoring system to obtain calibration images of a calibration sheet or other calibration object at various orientations and locations. The images are then stored and processed to calculate camera parameters defining the location and orientation of the image detector and identifying internal characteristics of the image detector, and the information are stored. The patient monitoring system can be re-calibrated by using the image detector to obtain an additional image of a calibration sheet or calibration object. The additional image and the stored camera parameters are then used to detect any apparent change in the internal characteristics of the image detector ( 10 ) (S 6 - 4 ).

CROSS REFERENCE TO RELATED APPLICATION

This application is a National Phase filing under 35 C.F.R. §371 of andclaims priority to PCT Patent Application No.: PCT/GB2015/053382, filedon Nov. 9, 2015, which claims the priority benefit under 35 U.S.C. §119of British Application No.: 1419936.8, filed on Nov. 10, 2014, thecontents of which are hereby incorporated in their entireties byreference.

BACKGROUND

Some embodiments relate to a method of calibrating a patient monitoringsystem for monitoring the location of a patient during radio therapy. Inparticular, some embodiments of the present invention concern a methodof calibrating a stereoscopic camera system for use with a radiotherapytreatment apparatus and the like where accurate positioning and thedetection of patient movement is important for successful treatment.

Radiotherapy involves projecting onto a predetermined region of apatient's body, a radiation beam so as to destroy or eliminate tumoursexisting therein. Such treatment is usually carried out periodically andrepeatedly. At each medical intervention, the radiation source must bepositioned with respect to the patient in order to irradiate theselected region with the highest possible accuracy to avoid radiatingadjacent tissue on which radiation beams would be harmful.

When applying radiation to a patient, the gating of treatment apparatusshould be matched with the breathing cycle so that radiation is focusedon the location of a tumour and collateral damage to other tissues isminimised. If movement of a patient is detected the treatment should behalted to avoid irradiating areas of a patient other than a tumourlocation.

For this reason a number of monitoring systems for assisting thepositioning of patients during radiotherapy have therefore been proposedsuch as those described in Vision RT's earlier patents and patentapplications U.S. Pat. No. 7,889,906, U.S. Pat. No. 7,348,974, U.S. Pat.No. 8,135,201, US2009/187112, WO2014/057280, and WO2015/008040 all ofwhich are hereby incorporated by reference.

In the systems described in Vision RT's patent applications,stereoscopic images of a patient are obtained and processed to generatedata identifying 3D positions of a large number of points correspondingto points on the surface of an imaged patient. Such data can be comparedwith data generated on a previous occasion and used to position apatient in a consistent manner or provide a warning when a patient movesout of position. Typically such a comparison involves undertakingProcrustes analysis to determine a transformation which minimises thedifferences in position between points on the surface of a patientidentified by data generated based on live images and points on thesurface of a patient identified by data generated on a previousoccasion.

Treatment plans for the application of radiotherapy are becomingincreasingly complex with treatment apparatus having multiple orfloating iso-centres. Also, there is an increasing trend to make use ofhigher doses of radiation during treatment in order to reduce overalltreatment time. Such increasing complexity and higher dosages bring withthem the increasing possibility of mistreatment. There is therefore anever increasing need for improvements in the accuracy of patientmonitoring systems.

To obtain a reasonable field of view, in a patient monitoring system,cameras monitoring a patient typically view a patient from a distance(e.g. 1 to 2 metres from the patient being monitored). Vision RT'spatient monitoring systems are able to generate highly accurate (e.g.sub millimeter) models of the surface of a patient. To do so, it isessential that the monitoring system is calibrated in order to establishcamera parameters identifying the relative locations and orientations ofthe image capture devices/cameras, any optical distortion caused by theoptical design of the lens of each image detector/camera e.g. barrel,pincushion, and moustache distortion and de-centring/tangentialdistortion, and other internal parameters of the cameras/image capturedevices (e.g. focal length, image centre, aspect ratio skew, pixelspacing etc.). Once known, camera parameters can be utilised tomanipulate obtained images to obtain images free of distortion. 3Dposition measurements can then be determined by matching correspondingportions in images obtained from different locations and deriving 3Dpositions from those matches and the relative locations and orientationsof the image capture devices/cameras.

Typically, calibration of a patient monitoring system involves capturingmultiple images of a calibration object of known size and with a knownarrangement of calibration markings at various orientations and variouslocations within the field of view, and processing the images usinginformation regarding the expected locations of the markings on thecalibration object to determine the various parameters. The accuracy ofthe calibration then very much depends on the number of images used inthe calibration process. The greater the number of images used, and thegreater the variation in the orientation of the calibration platebetween the various images, the more accurate the results. For example,the HALCON machine vision software package from MVTec Software GmbHrequires at least 10 to 15 images of a calibration plate, and mayrequire significantly more than this if the calibration object is smallrelative to the field of view of the image capture devices/cameras. As aconsequence of the number of images that are required to be captured, itcan be a time consuming to calibrate a computer vision system using suchprocesses.

When a computer vision system is used for monitoring the positioning ofpatients during radiotherapy treatment, a system needs to bere-calibrated very frequently (ideally for each individual patienttreatment session) to ensure that the parameters used to processcaptured images and generate computer models accurately reflect thecurrent relative locations of the stereoscopic cameras. The need forhigh accuracy and regular re-calibration is exacerbated in the case ofpatient monitoring system where patients are viewed from a distance asvery small changes in the orientations of the cameras can have asignificant impact on the accuracy of models. Due to this sensitivityvery regular calibration is particularly important for patientmonitoring systems particularly where cameras may accidentally beknocked or change orientation, for example in areas that are prone toearthquakes such as California or Japan small earthquake tremors couldcause movement of the image capture devices/cameras of the systemleading to errors in patient positioning and treatment. However, wheresuch calibration is undertaken at the desired frequency this has anadverse effect upon the throughput of patient treatment.

SUMMARY

It is therefore desirable to enhance the efficiency with whichcalibration procedures can be implemented for a patient monitoringsystem that is used with a radio therapy treatment apparatus.

In accordance with one aspect of the present invention there is provideda method of calibrating a patient monitoring system.

Initially, an image detector of the patient monitoring system is used toobtain a set of calibration images comprising images of a calibrationobject at various orientations and locations. These obtained calibrationimages are then stored and processed to calculate camera parametersdefining the location and orientation of the image detector andparameters identifying internal characteristics of the image detector(e.g. lens distortion, focal length, aspect ratio etc.).

Subsequently the patient monitoring system is re-calibrated by using theimage detector to obtain an additional image of the calibration object.This additional image and the stored camera parameters are then used todetect any apparent change in the internal characteristics of the imagedetector. If a change in the internal characteristics of an imagedetector is detected, a further set of calibration images using theimage detector are obtained and the camera parameters for the imagedetector are recalculated using the further set of calibration images.If no apparent change in the parameters internal to an image detector isdetected, the recalibration of the patient monitoring system is achievedby recalculating the camera parameters for the image detector using thestored set of calibration images and the additional image of thecalibration object obtained by the image detector.

The process described above provides a means for quickly confirming theaccuracy of a patient monitoring system that therefore facilitatesimproved efficiency of the calibration procedures. This is becauserather than performing a complete recalibration of the system based onmultiple images, only a single additional image is obtained and thecamera correction parameters are recalculated using this single newimage and the previously stored images from the original calibration.This reduces the amount of work necessary to perform the re-calibrationprocess and thus assists with maintaining throughput using the treatmentapparatus.

If internal characteristics of an image detector are adjusted then thesystem will no longer be accurate unless internal parameters areupdated. If any change in internal characteristics of an image detectorwhether actual or apparent can be detected this will indicate that afull re-calibration needs to be undertaken. In contrast, if cameraparameters relating to internal characteristics can be confirmed asaccurate then accurate camera parameters for the relative locations andorientation of an image detector can be determined based on theappearance of a calibration object of known dimensions in an image takenwith a camera and using the current internal parameters.

Detecting a change in the internal characteristics of an image detectorcan be achieved in a number of ways.

In some embodiments, detecting a change in internal characteristics ofan image detector may comprise comparing stored camera parametersdetermined using the stored calibration images and camera parametersdetermined using the stored calibration images and an additional image,and determining whether any of the parameters identifying internalcharacteristics of the image detector differ by more than a thresholdamount.

Alternatively in other embodiments, detecting an apparent change ininternal characteristics of an image detector may comprise processing anadditional image utilising the stored camera parameters to determine thelocation and orientation of the calibration object relative to the imagedetector, determining a back-projection of the expected appearance ofthe calibration object onto the image plane of the image detectorutilising the stored camera parameters and the determined location andorientation of the calibration sheet; and comparing the appearance ofthe calibration object within the additional image captured by the imagecapture detector with the back-projection of the expected appearance ofthe calibration object.

Recalculating the camera parameters for an image detector using thestored set of calibration images and an additional image of thecalibration object obtained by the image detector may comprisecalculating the location and orientation of the image detector andinternal characteristics of the image detector utilising the additionalimage as a base image relative to which the other images are compared.

In some embodiments, the calibration object may comprise a calibrationsheet and the additional image may comprise an image of the calibrationsheet located directly on a measurement plane of the patient monitoringsystem where the measurement plane of the patient monitoring system isdefined as z=0 plane of a global co-ordinate system defined for thepatient monitoring system. In such embodiments the global co-ordinatesystem defined for the patient monitoring system may be that used togenerate 3D wire mesh models of the surface of a patient monitored bythe system.

In some embodiments the set of calibration images used to calibrate thesystem may comprise at least 10 images of a calibration object atvarious orientations and various locations within a field of view of animage detector.

The method may be used to calibrate a patient monitoring systemcomprising a stereoscopic camera system having a plurality of imagedetectors, wherein each of the image detectors is calibrated in themanner described above.

The above described method of calibrating a patient monitoring systemmay be performed wherein the re-calibration is repeated periodically,with the monitoring system being used to monitor the position of apatient between obtaining images of the calibration object and there-calibration of the patient monitoring system.

BRIEF DESCRIPTION OF THE DRAWINGS

An embodiment of the present invention will now be described in greaterdetail with reference to the accompanying drawings in which:

FIG. 1 is a schematic perspective view of a patient monitor;

FIG. 2 is a front perspective view of the camera system of the patientmonitor of FIG. 1;

FIG. 3 is a schematic block diagram of the computer system of thepatient monitor of FIG. 1;

FIG. 4 is a plan view of a calibration sheet for use in the methodsdescribed herein;

FIGS. 5A-D are illustrative examples of images of the calibration sheetof FIG. 4 illustrating the results of a number of different imagedistortions; and

FIG. 6A-B is a flow diagram of a calibration method in accordance withthe present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Prior to describing a method of operating a patient monitoring systemfor use with a radio therapy treatment apparatus, a patient monitoringsystem and radiotherapy treatment apparatus will be described.

FIG. 1 is a schematic perspective view of an exemplary patientmonitoring system comprising a stereoscopic camera system 10 that isconnected by wiring (not shown) to a computer 14. The computer 14 isalso connected to treatment apparatus 16 such as a linear acceleratorfor applying radiotherapy. A mechanical couch 18 is provided as part ofthe treatment apparatus upon which a patient 20 lies during treatment.The treatment apparatus 16 and the mechanical couch 18 are arranged suchthat, under the control of the computer 14, the relative positions ofthe mechanical couch 18 and the treatment apparatus 16 may be varied,laterally, vertically, longitudinally and rotationally as is indicatedin the figure by the arrows adjacent the couch.

The treatment apparatus 16 comprises a main body 22 from which extends agantry 24. A collimator 26 is provided at the end of the gantry 24remote from the main body 22 of the treatment apparatus 16. To vary theangles at which radiation irradiates a patient 20, the gantry 24, underthe control of the computer 14, is arranged to rotate about an axispassing through the centre of the main body 22 of the treatmentapparatus 16. Additionally the location of irradiation by the treatmentapparatus may also be varied by rotating the collimator 26 at the end ofthe gantry 24.

In use, the stereoscopic cameras 10 obtain video images of a patient 20lying on the mechanical couch 18. These video images are passed via thewiring to the computer 14. The computer 14 then processes the images ofthe patient 20 to generate a model of the surface of the patient. Thismodel is compared with a model of the patient generated during earliertreatment sessions. When positioning a patient the difference between acurrent model surface and a target model surface obtained from anearlier session is identified and the positioning instructions necessaryto align the surfaces are determined and sent to the mechanical couch18. Subsequently during treatment any deviation from an initial set upcan be identified and if the deviation is greater than a threshold, thecomputer 14 sends instructions to the treatment apparatus 16 to causetreatment to be halted until a patient 20 can be repositioned.

FIG. 2 is a front perspective view of a stereoscopic camera system 10 ofthe patient monitoring system of FIG. 1.

In this embodiment the stereoscopic camera system 10 comprises a housing41 which is connected to a bracket 42 via a hinge 44. The bracket 42enables the stereoscopic camera system 10 to be attached in a fixedlocation to the ceiling of a treatment room whilst the hinge 44 permitsthe orientation of the stereoscopic camera system 10 to be orientatedrelative to the bracket 42 so that the stereoscopic camera system 10 isarranged to view a patient 20 on a mechanical couch 18.

A pair of lenses 46 are mounted at either end of the front surface 48 ofthe housing 41. These lenses 46 are positioned in front of image capturedevices/cameras such as CMOS active pixel sensors or charge coupleddevices (not shown) contained within the housing 41. The cameras/imagedetectors are arranged behind the lenses 46 so as to capture images of apatient 20 via the lenses 46.

A speckle projector 52 is provided in the middle of the front surface 48of the housing 41 between the two lenses 46. The speckle projector 52 isarranged to illuminate a patient 20 with a non-repeating speckledpattern of red light so that when images of a patient 20 are captured bythe two image detectors corresponding portions of captured images can bedistinguished. To that end the speckle projector comprises a lightsource such as a LED and a film with a random speckle pattern printed onthe film. In use light from the light source is projected via the filmand as a result a pattern consisting of light and dark areas isprojected onto the surface of a patient 20. When images of the projectedspeckle pattern are captured by the stereoscopic camera system 10 theimages can then be processed to determine the positions of a set ofpoints on the surface of the patient and hence the positioning of thepatient can be monitored.

FIG. 3 is a schematic block diagram of the computer 14 of the patientmonitor of FIG. 1.

In order for the computer 14 to process images received from thestereoscopic camera system 10, the computer 14 is configured by softwareeither provided on a disk 54 or by receiving an electrical signal 55 viaa communications network into a number of functional modules 56-64. Inthis example, the functional modules 56-64 comprise: a 3D positiondetermination module 56 for processing images received from thestereoscopic camera system 10, a model generation module 58 forprocessing data generated by the 3D position determination module 56 andconverting the data into a 3D wire mesh model of an imaged computersurface; a generated model store 60 for storing a 3D wire mesh model ofan imaged surface; a target model store 62 for storing a previouslygenerated 3D wire mesh model; and a matching module 64 for determiningrotations and translations required to match a generated model with atarget model.

In use, as images are obtained by the stereoscopic camera system 10,these images are processed by the 3D position determination module 56.This processing enables the 3D position determination module to identify3D positions of corresponding points in pairs of images on the surfaceof a patient 20. This is achieved by the 3D position determinationmodule 56 identifying corresponding points in pairs of images obtainedby the stereoscopic camera system 10 and then determining 3D positionsfor those points based on the relative positions of corresponding pointsin obtained pairs of images and stored camera parameters for each of theimage capture devices/cameras of the stereoscopic camera system 10.

Typically the identification of corresponding points is based onanalysis of image patches of around 16×16 pixels. In order to assistwith identifying and matching corresponding patches as has beendescribed the stereoscopic camera system 10 includes a speckle projector52 arranged to project a random or quasi random speckle pattern onto thepatient 20 being imaged so that different portions of the surface of thepatient 20 can be more easily distinguished. The size of the specklepattern is selected so that different patterns will be apparent indifferent image patches.

The position data generated by the 3D position determination module 56is then passed to the model generation module 58 which processes theposition data to generate a 3D wire mesh model of the surface of apatient 20 imaged by the stereoscopic cameras 10. In this embodiment the3D model comprises a triangulated wire mesh model where the vertices ofthe model correspond to the 3D positions determined by the 3D positiondetermination module 56. When such a model has been determined it isstored in the generated model store 60.

When a wire mesh model of the surface of a patient 20 has been stored,the matching module 64 is then invoked to determine a matchingtranslation and rotation between the generated model based on thecurrent images being obtained by the stereoscopic cameras 10 and apreviously generated model surface of the patient stored in the targetmodel store 62. The determined translation and rotation can then be sentas instructions to the mechanical couch 18 to cause the couch toposition the patient 20 in the same position relative to the treatmentapparatus 16 as they were when they were previously treated.

Subsequently, the stereoscopic cameras 10 can continue to monitor thepatient 20 and any variation in position can be identified by generatingfurther model surfaces and comparing those generated surfaces with thetarget model stored in the target model store 62. If it is determinedthat a patient has moved out of position, the treatment apparatus 16 canbe halted and the patient 20 repositioned, thereby avoiding irradiatingthe wrong parts of the patient 20.

In order to construct models of the surface of a patient with as greatan accuracy as possible, the patient monitoring system/stereoscopiccamera system needs to be calibrated so that matching portions of imagescan be converted into a determination of 3D positions. This involvesdetermining the relative positioning of the image capture devices.Additionally the calibration of the system must correct for any imagedistortion introduced by the lenses 46 or otherwise by the image capturedevices.

Conventional methods of calibrating a computer vision system involvecapturing a set of calibration images for each image capturedevice/camera that include multiple images of a calibration object ofknown size and with a known arrangement of calibration markings atvarious orientations and various locations within the field of view ofthe image capture device/camera. Such a calibration object typicallytakes the form of a calibration sheet, and FIG. 4 illustrates a planview of an exemplary calibration sheet 100 comprising a 70×70 cm sheetof flat rigid material such as aluminium or steel on which a patternrevealing a 34×32 matrix of markings/circles 101 a at known positions onthe surface of the sheet is provided. Additionally in this example,towards the centre of the calibration sheet are four smaller markers 101b adjacent to four circles the centres of which together identify thefour corners of a square of known size, and also a cross 102 formed by apair of dashed lines which meet at the centre of the sheet 100.

By way of further example, MVTec Software GmbH provide calibrationsheets in a range of sizes that are intended for use with their HALCONmachine vision software package, and these calibration sheets comprise a7×7 matrix of markings/circles.

FIGS. 5A-D illustrate a set of images of the calibration sheet of FIG.4. FIG. 5A illustrates an example of an image of the calibration sheet100 as viewed from an angle. FIG. 5B illustrates an example of an imageof the calibration sheet 100 in which barrel radial distortion ispresent, FIG. 5C illustrates an example of an image of the calibrationsheet 100 in which pincushion radial distortion is present, and FIG. 5Dillustrates an example of an image of the calibration sheet 100 in whichde-centring distortion is present.

If images of a calibration sheet 100 of known size and with a known setof markings are obtained by each image capture device/camera, parametersdefining the relative locations and orientations of the image capturedevices/cameras of the system can be derived, as can parameters definingthe internal characteristics of each of the image capturedevices/cameras (e.g. the focal length, radial distortion coefficients,tangential distortion coefficients etc.). Thus for example in the caseof FIG. 5A if the size and position of various markings of thecalibration sheet 100 are known, the relative orientation of the sheetand a camera obtaining an image such as is shown in FIG. 5A can bedetermined. If an affine transformation is applied to such an image toremove distortions arising due to viewing the calibration sheet 100 froman oblique angle, then the view of the calibration sheet 100 shouldcorrespond to the plan view shown in FIG. 4. To the extent that any ofthe array of markers does not correspond to the plan view of FIG. 4 dueto distortions arising from the internal characteristics of the imagecapture device/camera, such as are illustrated in FIGS. 5B-D, then thesecan be identified so that appropriate corrections for these distortionscan be made before processing images to generate 3D models based oncaptured images.

FIGS. 6A and 6B are a flow diagram illustrating the improved process ofcalibrating and operating a patient monitoring system/stereoscopiccamera system in accordance with the present invention.

Initially, the patient monitoring system/stereoscopic camera systemundergoes a full calibration, as illustrated in FIG. 6A, to establishcamera parameters for each of the image capture devices/cameras of thesystem.

In FIG. 6A, in the first step of this full calibration of an imagecapture device/camera (s6-1), the image capture device/camera is used toobtain and store a set of calibration images for the image capturedevice/camera that includes multiple images of a calibration sheet witha known arrangement of calibration markings at various orientations andvarious locations within the field of view of the image capturedevice/camera.

This set of calibration images comprises a number of images (e.g. 10 to15 different images) of the calibration sheet at different positions andorientations within the field of view of the image capture device/camerabeing calibrated. One of these images is designated as a base image forthe calibration process and the surface of the calibration sheet in thebase image is used to define a plane z=0 of a global co-ordinate systemdefined for the system.

The set of calibration images of the calibration sheet, including thebase image, are then processed using information regarding the expectedlocations of the markings on the calibration sheet to determine cameraparameters for the image capture device/camera that captured the set ofcalibration images and these camera parameters are then stored (S6-2).

The full calibration procedure illustrated in FIG. 6A is undertaken foreach of the image capture devices/cameras until camera parameters havebeen calculated and stored for all of the image capture devices/camerasof the system. This determination of camera parameters for each imagecapture device/camera is done using conventional techniques.

By way of example, each of the calibration images within a set can beprocessed to extract the coordinates within the image of the markings onthe calibration sheet. These coordinates can then be compared with theexpected locations of the markings in the coordinate system to determinethe parameters identifying the relative position and orientation ofcameras and parameters internal to the cameras themselves (e.g. lensdistortion, focal length, aspect ratio etc.), as is described in detailin “A Versatile Camera Calibration Technique for High-Accuracy 3DMachine Vision Metrology Using Off the Shelf TV Cameras and Lenses”,Roger Tsai, IEEE Journal of Robotics and Automation, Vol. Ra-3, No. 4,August 1987 which is hereby incorporated by reference. As a furtherexample, the HALCON machine vision software package from MVTec SoftwareGmbH implements an alternative technique for determining a set of cameraparameters for a camera from a set of calibration images, as isdescribed in “Solution Guide III-C—3D Vision” issued by MVTec SoftwareGmbH.

The stored camera parameters can then be used by the patient monitoringsystem when monitoring a patient to process the images captured by theimage capture device/camera (e.g. to remove distortions from images andto convert matches between portions of different images into 3D surfacemeasurements based upon the locations of relative matches in images andthe relative locations and orientations of the cameras used to capturethe images).

At some time after the full calibration has been completed, are-calibration is initiated. Such an initiation of the re-calibrationcould be a manual action in which the re-calibration is initiated by thesystem operator at the start of the day or even between patients.

FIG. 6B is a flow diagram illustrating this re-calibration procedure,during which each image capture device/camera of the system undergoes adegree of re-calibration

In a first step of the re-calibration procedure, the image capturedevice/camera undergoing re-calibration is used to obtain an additionalimage of a calibration sheet (S6-3), with the calibration sheetpreferably placed at or close to a location to define the plane z=0 of aglobal co-ordinate system defined for the system. The placement of thecalibration sheet 100 at the desired location can be achieved by placingthe calibration sheet 100 on the surface of the mechanical couch 18 andlocating the sheet with the centre of the sheet 100 located at theiso-centre of the treatment apparatus 16. In many cases the accurateplacement of the sheet can be facilitated using laser light to highlightthe location of the iso-centre as is often used in conventionalradiotherapy apparatus and positioning and aligning the sheet 100 andthe cross 102 on the sheet 100 with the projected planes of laser light.This additional image is then used to implement a calibration check toconfirm whether or not the stored internal camera parameters for theimage capture device/camera (i.e. the camera parameters relating to theinternal characteristics of the camera) are still valid/accurate (S6-4)which may not be the case if for example the cameras have been knockedor otherwise have move out of position.

There are a number of possible methods by which an additional image canbe used to confirm whether or not the stored internal camera parametersfor an image capture device/camera are still valid/accurate.

By way of example, in order to check the accuracy of the previouslystored camera parameters, the additional image can be processed usingthe previously stored camera parameters identifying the location andorientation of the image detectors and the internal cameracharacteristics to determine the location and orientation of thecalibration sheet relative to the image detector. A back-projection ofthe markings on the calibration sheet onto the image plane can then beimplemented using the determined location and orientation of thecalibration sheet and the stored camera parameters. The accuracy of thestored internal camera parameters for a particular image capturedevice/camera can then be determined by comparing the appearance of themarkings on the calibration sheet within the additional image capturedby the image capture device/camera with the appearance of thecalibration object in the back-projection. This comparison will thenreveal any changes in the internal characteristics of the imagedetector. The back-projection would make use of data describing thecalibration object and the locations of the marking on the calibrationobject.

As an alternative, the accuracy of the previously stored cameraparameters relating to internal aspects of the cameras (e.g. lensdistortion, focal length, aspect ratio etc.), could be determined byre-calculating these camera parameters using both the additional imageand the previously stored images from the set of calibration images usedin the initial calibration calculation. The re-calculated cameraparameters relating to internal aspects of the image capturedevice/camera can then be compared with the corresponding stored cameraparameters relating to these internal aspects that were calculated forthe image capture device/camera during the initial full calibrationprocess to determine if there is a substantial difference between there-calculated parameters and the stored parameters, and thereforewhether or not the stored camera parameters relating to the internalaspects of the cameras are still accurate.

For example, this could involve calculating the difference between thevalues of each of the re-calculated camera parameters relating tointernal aspects of the camera and the values of the correspondingstored internal camera parameters, and determining if any of thesedifferences exceed a corresponding threshold.

If it is determined that the stored camera parameters relating tointernal characteristics of the camera resulting from the last fullcalibration are still sufficiently accurate (i.e. that the detectedchange in the camera parameters or that the differences between imagedand projected positions for markings on a calibration sheet differ byless than a threshold amount), then the image capture device/camera canbe partially re-calibrated, without having to undertake a fullcalibration process that would require an entirely new set ofcalibration images. This can be achieved by replacing the previouslystored base image with the newly captured image (S6-8), and thenre-calculating the camera parameters using the stored calibrationimages, which now include the newly captured image (S6-9).

This re-calibration can be achieved without having to obtain acompletely new set of calibration images as if it is determined that thecamera parameters relating to internal aspects of the cameras are stillsufficiently accurate (i.e. that none of the cameras internalcharacteristics have appear to have changed significantly) the storedimages can be reused together with the new base image to calibrate thesystem. This is even the case if the cameras have been knocked out ofposition in the period of time between obtaining the original set ofcamera images (s6-1) and the new additional image (s6-3). This isbecause any movement of the image capture device/camera can be takeninto account by using the new image as the basis for determining thecurrent location and orientation of the cameras.

If it is determined that the internal characteristics of the imagedetector appear to have changed significantly, then a full recalibrationwill be required. For example, this could be because some internalaspect of the camera has been adjusted intentionally or unintentionally.When this occurs, the system therefore determines that a fullrecalibration of this image capture device/camera is required (S6-10)and indicates this to the operator. The process therefore proceeds toimplement a full recalibration (i.e. returns to step S6-1) requiringfull recalibration using an entirely new set of calibration images.

When a re-calibration has been initiated, the procedure illustrated inFIG. 6B is repeated until camera parameters have been re-calculated andstored for all of the image capture devices/cameras of the system. Thesystem then awaits the initiation of the next re-calibration check.

In the above described embodiment of the calibration process, theinitial full calibration, the calibration check, and the full andpartial recalibration of the patient monitoring system are described asmaking use of images of a calibration object in the form of acalibration sheet 100. Whilst each of these steps could be implementedusing the same calibration object, or at least a calibration object ofthe same design, it is also equally possible to make use of a differentform/design of calibration object for each step, provided that theform/design of the calibration object is sufficient to allow the 3Dlocation and orientation of the calibration object to be determined andprovided that the system has data describing each calibration object.For example, a first calibration object could be used when obtaining theset of calibration images for the initial full calibration, a secondcalibration object could then be used when obtaining the additionalimage for the calibration check (with the additional image of thissecond calibration object also being used for a partial recalibration),and a third calibration object could then be used when obtaining thefurther set of calibration images for a full recalibration.

In a particularly advantageous embodiment, a first calibration object isused to implement the initial full calibration, whilst a secondcalibration object is used to implement the calibration check and anypartial recalibration. The first calibration object should then bereused should a full recalibration be required. This approach can thenmake use of a first calibration object that is more expensive than thesecond calibration object, with the first calibration object being madetemporarily available by the manufacturer/supplier of the patientmonitoring system as and when required for infrequent full calibrations,and the second calibration object being supplied to the end user of thepatient monitoring system for the more regular partial recalibrations.

Although the embodiments of the invention described with reference tothe drawings comprise computer apparatus and processes performed incomputer apparatus, the invention also extends to computer programs,particularly computer programs on or in a carrier, adapted for puttingthe invention into practice. The program may be in the form of source orobject code or in any other form suitable for use in the implementationof the processes according to the invention. The carrier can be anyentity or device capable of carrying the program.

For example, the carrier may comprise a storage medium, such as a ROM,for example a CD ROM or a semiconductor ROM, or a magnetic recordingmedium, for example a floppy disc or hard disk. Further, the carrier maybe a transmissible carrier such as an electrical or optical signal whichmay be conveyed via electrical or optical cable or by radio or othermeans. When a program is embodied in a signal which may be conveyeddirectly by a cable or other device or means, the carrier may beconstituted by such cable or other device or means. Alternatively, thecarrier may be an integrated circuit in which the program is embedded,the integrated circuit being adapted for performing, or for use in theperformance of, the relevant processes.

1. A method of calibrating a patient monitoring system the methodcomprising: using an image detector of the patient monitoring system toobtain a set of calibration images comprising images of a calibrationobject at various orientations and locations; storing the obtainedcalibration images; processing the stored calibration images tocalculate camera parameters defining the location and orientation of theimage detector and parameters identifying internal characteristics ofthe image detector, and storing the calculated parameters; andsubsequently re-calibrating the patient monitoring system by: obtainingan additional image of a calibration object using the image detector;detecting an apparent change in the internal characteristics of theimage detector using the additional image obtained using the imagedetector and the stored camera parameters, and if an apparent change inthe internal characteristics of an image detector is detected, obtaininga further set of calibration images using the image detector andrecalculating the camera parameters for the image detector using thefurther set of calibration images; and if no apparent change in theinternal characteristics of an image detector is detected, recalculatingthe camera parameters for the image detector using the stored set ofcalibration images and the additional image of the calibration objectobtained by the image detector.
 2. The method of claim 1, whereindetecting an apparent change in internal characteristics of an imagedetector comparing the calculated camera parameters determined using thestored calibration images and camera parameters determined using thestored calibration images and the additional image, and determiningwhether any of the parameters identifying internal characteristics ofthe image detector differ by more than a threshold amount.
 3. The methodof claim 1, wherein detecting an apparent change in internalcharacteristics of an image detector comprises: processing theadditional image utilising the stored camera parameters to determine thelocation and orientation of the calibration object relative to the imagedetector, determining a back-projection of the expected appearance ofthe calibration object onto the image plane of the image detectorutilising the stored camera parameters and the determined location andorientation of the calibration sheet; and comparing the appearance ofthe calibration object within the additional image captured by the imagecapture detector with the back-projection of the expected appearance ofthe calibration object.
 4. The method of claim 1, wherein, recalculatingthe camera parameters for an image detector using the stored set ofcalibration images and the additional image of the calibration objectobtained by the image detector comprises calculating the location andorientation of the image detector and internal characteristics of theimage detector utilising the additional image as a base image relativeto which the other images are compared.
 5. The method of claim 1,wherein the calibration object used when obtaining the set ofcalibration images differs from the calibration object used to obtainthe additional image.
 6. The method of claim 5, wherein the calibrationobject used when obtaining a further set of calibration images is thesame as the calibration object used when obtaining the set ofcalibration images.
 7. The method of claim 1, wherein the or eachcalibration object comprises a calibration sheet.
 8. The method of claim7, wherein the additional image comprises an image of a calibrationsheet located directly on a measurement plane of the patient monitoringsystem.
 9. The method of claim 8, wherein the measurement plane of thepatient monitoring system is defined as z=0 plane of a globalco-ordinate system defined for the patient monitoring system.
 10. Themethod of claim 9, wherein the global co-ordinate system defined for thepatient monitoring system is that used to generate 3D wire mesh modelsof the surface of a patient monitored by the system.
 11. The method ofclaim 1, wherein a set of calibration images comprises at least 10images of a calibration object at various orientations and variouslocations within a field of view of an image detector.
 12. The method ofclaim 1, wherein the patient monitoring system comprises a stereoscopiccamera system comprising a plurality of image detectors, whereincalibrating a patient monitoring system comprises calibrating each ofthe image detectors in accordance with the method of any precedingclaim.
 13. The method of claim 1, wherein the re-calibration of thepatient monitoring system is repeated periodically.
 14. The method ofclaim 13, further comprising using the patient monitoring system tomonitor the position of a patient between obtaining images of thecalibration object and the re-calibration of the patient monitoringsystem.